{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 皮马印第安人糖尿病分类\n",
    "请在 Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集）进行分类器练 习。\n",
    "\n",
    "原始数据集地址：https://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes  \n",
    "数据集只有一个文件（diabetes.csv）：Pima Indians Diabetes Dataset 包括根据医疗 记录的比马印第安人 5 年内糖尿病的发病情况，这是一个两类分类问题。每个类的 样本数目数量不均等。一共有 768 个样本，每个样本有 8 个输入变量和 1 个输出 变量。缺失值通常用零值编码。 \n",
    " \n",
    "1) 字段说明 Pregnancies： 怀孕次数 Glucose： 口服葡萄糖耐受试验中，2 小时的血浆葡萄糖浓度。 BloodPressure： 舒张压（mm Hg） SkinThickness： 三头肌皮肤褶层厚度（mm） Insulin：2 小时血清胰岛素含量（μU/ ml） BMI： 体重指数（体重，kg /（身高，m）^ 2） 2) DiabetesPedigreeFunction： 糖尿病家族史 3) Age： 年龄（岁） Outcome： 输出变了/类别标签（0 或 1，出现糖尿病为 1, 否则为 0） \n",
    " \n",
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "#模型\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "#评价指标\n",
    "from sklearn.metrics import log_loss, accuracy_score, f1_score\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据读取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pregnancies</th>\n",
       "      <th>Glucose</th>\n",
       "      <th>BloodPressure</th>\n",
       "      <th>SkinThickness</th>\n",
       "      <th>Insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>DiabetesPedigreeFunction</th>\n",
       "      <th>Age</th>\n",
       "      <th>Outcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6</td>\n",
       "      <td>148</td>\n",
       "      <td>72</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>33.6</td>\n",
       "      <td>0.627</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>85</td>\n",
       "      <td>66</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>26.6</td>\n",
       "      <td>0.351</td>\n",
       "      <td>31</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8</td>\n",
       "      <td>183</td>\n",
       "      <td>64</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>23.3</td>\n",
       "      <td>0.672</td>\n",
       "      <td>32</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>89</td>\n",
       "      <td>66</td>\n",
       "      <td>23</td>\n",
       "      <td>94</td>\n",
       "      <td>28.1</td>\n",
       "      <td>0.167</td>\n",
       "      <td>21</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>137</td>\n",
       "      <td>40</td>\n",
       "      <td>35</td>\n",
       "      <td>168</td>\n",
       "      <td>43.1</td>\n",
       "      <td>2.288</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Pregnancies  Glucose  BloodPressure  SkinThickness  Insulin   BMI  \\\n",
       "0            6      148             72             35        0  33.6   \n",
       "1            1       85             66             29        0  26.6   \n",
       "2            8      183             64              0        0  23.3   \n",
       "3            1       89             66             23       94  28.1   \n",
       "4            0      137             40             35      168  43.1   \n",
       "\n",
       "   DiabetesPedigreeFunction  Age  Outcome  \n",
       "0                     0.627   50        1  \n",
       "1                     0.351   31        0  \n",
       "2                     0.672   32        1  \n",
       "3                     0.167   21        0  \n",
       "4                     2.288   33        1  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv(\"diabetes.csv\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 768 entries, 0 to 767\n",
      "Data columns (total 9 columns):\n",
      "Pregnancies                 768 non-null int64\n",
      "Glucose                     768 non-null int64\n",
      "BloodPressure               768 non-null int64\n",
      "SkinThickness               768 non-null int64\n",
      "Insulin                     768 non-null int64\n",
      "BMI                         768 non-null float64\n",
      "DiabetesPedigreeFunction    768 non-null float64\n",
      "Age                         768 non-null int64\n",
      "Outcome                     768 non-null int64\n",
      "dtypes: float64(2), int64(7)\n",
      "memory usage: 54.1 KB\n"
     ]
    }
   ],
   "source": [
    "data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pregnancies</th>\n",
       "      <th>Glucose</th>\n",
       "      <th>BloodPressure</th>\n",
       "      <th>SkinThickness</th>\n",
       "      <th>Insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>DiabetesPedigreeFunction</th>\n",
       "      <th>Age</th>\n",
       "      <th>Outcome</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "      <td>768.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.845052</td>\n",
       "      <td>120.894531</td>\n",
       "      <td>69.105469</td>\n",
       "      <td>20.536458</td>\n",
       "      <td>79.799479</td>\n",
       "      <td>31.992578</td>\n",
       "      <td>0.471876</td>\n",
       "      <td>33.240885</td>\n",
       "      <td>0.348958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>3.369578</td>\n",
       "      <td>31.972618</td>\n",
       "      <td>19.355807</td>\n",
       "      <td>15.952218</td>\n",
       "      <td>115.244002</td>\n",
       "      <td>7.884160</td>\n",
       "      <td>0.331329</td>\n",
       "      <td>11.760232</td>\n",
       "      <td>0.476951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.078000</td>\n",
       "      <td>21.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>62.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>27.300000</td>\n",
       "      <td>0.243750</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.000000</td>\n",
       "      <td>117.000000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>23.000000</td>\n",
       "      <td>30.500000</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>0.372500</td>\n",
       "      <td>29.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>140.250000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>32.000000</td>\n",
       "      <td>127.250000</td>\n",
       "      <td>36.600000</td>\n",
       "      <td>0.626250</td>\n",
       "      <td>41.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>17.000000</td>\n",
       "      <td>199.000000</td>\n",
       "      <td>122.000000</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>846.000000</td>\n",
       "      <td>67.100000</td>\n",
       "      <td>2.420000</td>\n",
       "      <td>81.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Pregnancies     Glucose  BloodPressure  SkinThickness     Insulin  \\\n",
       "count   768.000000  768.000000     768.000000     768.000000  768.000000   \n",
       "mean      3.845052  120.894531      69.105469      20.536458   79.799479   \n",
       "std       3.369578   31.972618      19.355807      15.952218  115.244002   \n",
       "min       0.000000    0.000000       0.000000       0.000000    0.000000   \n",
       "25%       1.000000   99.000000      62.000000       0.000000    0.000000   \n",
       "50%       3.000000  117.000000      72.000000      23.000000   30.500000   \n",
       "75%       6.000000  140.250000      80.000000      32.000000  127.250000   \n",
       "max      17.000000  199.000000     122.000000      99.000000  846.000000   \n",
       "\n",
       "              BMI  DiabetesPedigreeFunction         Age     Outcome  \n",
       "count  768.000000                768.000000  768.000000  768.000000  \n",
       "mean    31.992578                  0.471876   33.240885    0.348958  \n",
       "std      7.884160                  0.331329   11.760232    0.476951  \n",
       "min      0.000000                  0.078000   21.000000    0.000000  \n",
       "25%     27.300000                  0.243750   24.000000    0.000000  \n",
       "50%     32.000000                  0.372500   29.000000    0.000000  \n",
       "75%     36.600000                  0.626250   41.000000    1.000000  \n",
       "max     67.100000                  2.420000   81.000000    1.000000  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看下有没有不合理数据\n",
    "data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br/>Glucose（血糖浓度）</br>\n",
    "<br/>BloodPressure（舒张压）</br>\n",
    "<br/>SkinThickness( 三头肌皮肤褶层厚度)</br>\n",
    "<br/>Insulin(血清胰岛素含量)</br>\n",
    "<br/>BMI(体重)</br>\n",
    "\n",
    "这些指标在医学等于0没有意义，是缺失数据补的0，要对这些缺失值进行填补"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Glucose            5\n",
      "BloodPressure     35\n",
      "SkinThickness    227\n",
      "Insulin          374\n",
      "BMI               11\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "NaN_col = ['Glucose','BloodPressure', 'SkinThickness', 'Insulin', 'BMI']\n",
    "print((data[NaN_col] == 0).sum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'Number of occurences')"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAE7ZJREFUeJzt3X/wXXV95/Hni5/iL8KPyLD50WCN29pRKY0sO+zUFqwjuCWUiq3DCsV0053FLlu73aJ1Zbb0h1qVlrHjbnYRg2WhwCIEFktpBJ3+QAkqAUrdhJRCNiwJioCwYJH3/nE/X3MNJ9+ckNzvvXy/z8fMd+75fO7n3Pu+TvTlOZ9zPidVhSRJO9pn3AVIkiaTASFJ6mRASJI6GRCSpE4GhCSpkwEhSepkQEiSOhkQkqROBoQkqdN+4y5gTxx++OG1ZMmScZchSS8qd9xxxyNVNX9X417UAbFkyRLWrVs37jIk6UUlyT/0GecpJklSJwNCktTJgJAkdTIgJEmdDAhJUqeRBkSS+5PcleTrSda1vkOT3JxkQ3s9pPUnyUVJNiZZn+SYUdYmSZreTBxB/HRVHV1Vy1r7PGBtVS0F1rY2wEnA0va3EvjUDNQmSdqJcZxiWg6sbturgVOH+i+tgduAeUmOHEN9kiRGHxAF/HmSO5KsbH1HVNVDAO31Va1/AfDg0L6bW58kaQxGfSf18VW1JcmrgJuT/N00Y9PRV88bNAialQCLFy/e4wJ/4jcu3ePP0Oxzxx+cOe4SpLEb6RFEVW1pr1uBzwHHAg9PnTpqr1vb8M3AoqHdFwJbOj5zVVUtq6pl8+fvcikRSdILNLKASPKyJK+Y2gbeCtwNrAHOasPOAq5r22uAM9vVTMcBj02dipIkzbxRnmI6Avhckqnv+R9V9WdJbgeuTLICeAA4vY2/ETgZ2Ag8BZw9wtokSbswsoCoqk3AGzv6vwmc2NFfwDmjqkeStHu8k1qS1MmAkCR1MiAkSZ0MCElSJwNCktTJgJAkdTIgJEmdDAhJUicDQpLUyYCQJHUyICRJnQwISVInA0KS1MmAkCR1MiAkSZ0MCElSJwNCktTJgJAkdTIgJEmdDAhJUicDQpLUyYCQJHUyICRJnQwISVInA0KS1MmAkCR1MiAkSZ0MCElSJwNCktTJgJAkdTIgJEmdDAhJUqeRB0SSfZN8LckNrX1Uki8n2ZDkT5Mc0PoPbO2N7f0lo65NkrRzM3EEcS5w71D7I8CFVbUUeBRY0fpXAI9W1WuAC9s4SdKYjDQgkiwE3g7899YOcAJwdRuyGji1bS9vbdr7J7bxkqQxGPURxB8C/xF4rrUPA75dVc+29mZgQdteADwI0N5/rI2XJI3ByAIiyb8EtlbVHcPdHUOrx3vDn7syybok67Zt27YXKpUkdRnlEcTxwClJ7geuYHBq6Q+BeUn2a2MWAlva9mZgEUB7/2DgWzt+aFWtqqplVbVs/vz5Iyxfkua23QqIJIckeUOfsVX1/qpaWFVLgF8EvlBVZwC3AO9ow84Crmvba1qb9v4Xqup5RxCSpJmxy4BIcmuSVyY5FLgTuCTJJ/bgO38TeF+SjQzmGC5u/RcDh7X+9wHn7cF3SJL20H67HsLBVfV4kl8GLqmq85Os350vqapbgVvb9ibg2I4xTwOn787nSpJGp88ppv2SHAm8E7hhxPVIkiZEn4D4beAm4L6quj3Jq4ENoy1LkjRuuzzFVFVXAVcNtTcBPz/KoiRJ49dnkvq1SdYmubu135Dkg6MvTZI0Tn1OMf034P3APwJU1XoGl61KkmaxPgHx0qr6yg59z3aOlCTNGn0C4pEkP0xb9iLJO4CHRlqVJGns+twHcQ6wCviRJP8H+HvgX420KknS2PW5imkT8JYkLwP2qaonRl+WJGnc+lzF9HtJ5lXVk1X1RFuP6XdmojhJ0vj0mYM4qaq+PdWoqkeBk0dXkiRpEvQJiH2THDjVSHIQcOA04yVJs0CfSeo/AdYmuYTBlUzvYfujQSVJs1SfSeqPJrkLOJHBU98uqKqbRl6ZJGms+hxBUFWfBz4/4lokSROkz1VMpyXZkOSxJI8neSLJ4zNRnCRpfPocQXwU+NmqunfUxUiSJkefq5geNhwkae7pcwSxLsmfAtcCz0x1VtU1I6tKkjR2fQLilcBTwFuH+gowICRpFutzmevZM1GIJGmy+EQ5SVInnygnSerkE+UkSZ18opwkqdMLfaLcGSOtSpI0dtMGRJJ9gGVV5RPlJGmOmfYUU1U9B7y3bT9pOEjS3NFnDuLmJP8hyaIkh079jbwySdJY9ZmDeE97PWeor4BX7/1yJEmTos+d1EfNRCGSpMmyy4BIcmZXf1VduvfLkSRNij6nmN40tP0SBo8e/SpgQEjSLNbnFNOvDreTHAx8dlf7JXkJ8CXgwPY9V1fV+UmOAq4ADmUQNO+uqu8mOZBB6PwE8E3gF6rq/t37OZKkvaXPVUw7egpY2mPcM8AJVfVG4GjgbUmOAz4CXFhVS4FHgRVt/Arg0ap6DXBhGydJGpM+q7len2RN+7sB+AZw3a72q4HvtOb+7a+AE4CrW/9q4NS2vby1ae+fmCS9f4kkaa/qMwfxsaHtZ4F/qKrNfT48yb7AHcBrgD8G7gO+XVVTi/1tBha07QXAgwBV9WySx4DDgEf6fJckae/qExAPAA9V1dMASQ5KsqTP/EBVfQ84Osk84HPAj3YNa69dRwu1Y0eSlcBKgMWLF/coX5L0QvSZg7gKeG6o/b3W11tVfRu4FTgOmJdkKpgWAlva9mZgEUB7/2DgWx2ftaqqllXVsvnz5+9OGZKk3dAnIParqu9ONdr2AbvaKcn8duRAkoOAtwD3ArcA72jDzmL7fMaa1qa9/4Wqet4RhCRpZvQ5xbQtySlVtQYgyXL6zQscCaxu8xD7AFdW1Q1J/ha4IsnvAF8DLm7jLwY+m2QjgyMHn1onSWPUJyD+DXBZkk+29mag8+7qYe3RpD/e0b8JOLaj/2ng9B71SJJmQJ8b5e4DjkvyciAu+S1Jc0Of+yB+L8m8qvpOVT2R5JB2ekiSNIv1maQ+qV2FBEBVPQqcPLqSJEmToE9A7NvWSQK+f0XSgdOMlyTNAn0mqf8EWJvkEgY3rr2H7UtiSJJmqT6T1B9Nsp7BfQwAF1TVTaMtS5I0bn2OIGBwv8LUYntfG105kqRJ0ecqpncCX2Fwd/M7gS8necf0e0mSXuz6HEH8FvCmqtoKgyU0gL9g+5Ldkkbggd9+/bhL0ARa/KG7Zuy7+lzFtM9UODTf7LmfJOlFrM8RxJ8luQm4vLV/AbhxdCVJkiZBn6uYfiPJacC/YPDMhlVV9bmRVyZJGqteVzFV1TXANSOuRZI0QZxLkCR1MiAkSZ12GhBJ1rbXj8xcOZKkSTHdHMSRSd4MnJLkCgYT1N9XVV8daWWSpLGaLiA+BJwHLAQ+scN7BZwwqqIkSeO304CoqquBq5P8p6q6YAZrkiRNgD73QVyQ5BTgJ1vXrVV1w2jLkiSNW5/F+n4fOBf42/Z3buuTJM1ifW6UeztwdFU9B5BkNYMlv98/ysIkSePV9z6IeUPbB4+iEEnSZOlzBPH7wNeS3MLgUtefxKMHSZr1+kxSX57kVuBNDALiN6vq/466MEnSePVdrO8hYM2Ia5EkTRDXYpIkdTIgJEmdpg2IJPskuXumipEkTY5pA6Ld+3BnksUzVI8kaUL0maQ+ErgnyVeAJ6c6q+qUkVUlSRq7PgHxn0dehSRp4uxykrqqvgjcD+zftm8HdvksiCSLktyS5N4k9yQ5t/UfmuTmJBva6yGtP0kuSrIxyfokx+zRL5Mk7ZE+i/X9a+Bq4L+2rgXAtT0++1ng16vqR4HjgHOSvI7BMybWVtVSYG1rA5wELG1/K4FP7cbvkCTtZX0ucz0HOB54HKCqNgCv2tVOVfXQ1FPnquoJ4F4G4bIcWN2GrQZObdvLgUtr4DZgXpIjd+O3SJL2oj4B8UxVfXeqkWQ/Bk+U6y3JEuDHgS8DR7Q7s6fu0J4KmwXAg0O7bW59kqQx6BMQX0zyAeCgJD8DXAVc3/cLkrwc+J/Av6+qx6cb2tH3vCBKsjLJuiTrtm3b1rcMSdJu6hMQ5wHbgLuAXwFuBD7Y58OT7M8gHC6rqmta98NTp47a69bWvxlYNLT7QmDLjp9ZVauqallVLZs/f36fMiRJL0Cf1Vyfaw8J+jKD/0f/jara5SmmJAEuBu6tqk8MvbUGOAv4cHu9bqj/vUmuAP4Z8NjUqShJ0szbZUAkeTvwX4D7GJwGOirJr1TV53ex6/HAu4G7kny99X2AQTBcmWQF8ABwenvvRuBkYCPwFHD2bv4WSdJe1OdGuY8DP11VGwGS/DDwv4BpA6Kq/pLueQWAEzvGF4MrpiRJE6DPHMTWqXBoNrF93kCSNEvt9AgiyWlt854kNwJXMpiDOJ3B3dSSpFlsulNMPzu0/TDw5ra9DThkZBVJkibCTgOiqpwklqQ5rM9VTEcBvwosGR7vct+SNLv1uYrpWgb3M1wPPDfaciRJk6JPQDxdVReNvBJJ0kTpExB/lOR84M+BZ6Y6p1ZqlSTNTn0C4vUM7og+ge2nmKq1JUmzVJ+A+Dng1cNLfkuSZr8+d1LfCcwbdSGSpMnS5wjiCODvktzOD85BeJmrJM1ifQLi/JFXIUmaOH2eB/HFmShEkjRZ+txJ/QTbH/15ALA/8GRVvXKUhUmSxqvPEcQrhttJTgWOHVlFkqSJ0Ocqph9QVdfiPRCSNOv1OcV02lBzH2AZ2085SZJmqT5XMQ0/F+JZ4H5g+UiqkSRNjD5zED4XQpLmoOkeOfqhafarqrpgBPVIkibEdEcQT3b0vQxYARwGGBCSNItN98jRj09tJ3kFcC5wNnAF8PGd7SdJmh2mnYNIcijwPuAMYDVwTFU9OhOFSZLGa7o5iD8ATgNWAa+vqu/MWFWSpLGb7ka5Xwf+CfBBYEuSx9vfE0ken5nyJEnjMt0cxG7fZS1Jmj0MAUlSJwNCktTJgJAkdTIgJEmdDAhJUicDQpLUaWQBkeTTSbYmuXuo79AkNyfZ0F4Paf1JclGSjUnWJzlmVHVJkvoZ5RHEZ4C37dB3HrC2qpYCa1sb4CRgaftbCXxqhHVJknoYWUBU1ZeAb+3QvZzBmk6011OH+i+tgduAeUmOHFVtkqRdm+k5iCOq6iGA9vqq1r8AeHBo3ObW9zxJViZZl2Tdtm3bRlqsJM1lkzJJnY6+zudeV9WqqlpWVcvmz58/4rIkae6a6YB4eOrUUXvd2vo3A4uGxi0EtsxwbZKkITMdEGuAs9r2WcB1Q/1ntquZjgMemzoVJUkaj2kfGLQnklwO/BRweJLNwPnAh4Erk6wAHgBOb8NvBE4GNgJPMXhynSRpjEYWEFX1rp28dWLH2ALOGVUtkqTdNymT1JKkCWNASJI6GRCSpE4GhCSpkwEhSepkQEiSOhkQkqROBoQkqZMBIUnqZEBIkjoZEJKkTgaEJKmTASFJ6mRASJI6GRCSpE4GhCSpkwEhSepkQEiSOhkQkqROBoQkqZMBIUnqZEBIkjoZEJKkTgaEJKmTASFJ6mRASJI6GRCSpE4GhCSpkwEhSepkQEiSOhkQkqROBoQkqdNEBUSStyX5RpKNSc4bdz2SNJdNTEAk2Rf4Y+Ak4HXAu5K8brxVSdLcNTEBARwLbKyqTVX1XeAKYPmYa5KkOWuSAmIB8OBQe3PrkySNwX7jLmBIOvrqeYOSlcDK1vxOkm+MtKq55XDgkXEXMQnysbPGXYJ+kP82p5zf9T+Vu+2H+gyapIDYDCwaai8Etuw4qKpWAatmqqi5JMm6qlo27jqkHflvczwm6RTT7cDSJEclOQD4RWDNmGuSpDlrYo4gqurZJO8FbgL2BT5dVfeMuSxJmrMmJiAAqupG4MZx1zGHeepOk8p/m2OQqufNA0uSNFFzEJKkCWJAyCVONLGSfDrJ1iR3j7uWuciAmONc4kQT7jPA28ZdxFxlQMglTjSxqupLwLfGXcdcZUDIJU4kdTIg1GuJE0lzjwGhXkucSJp7DAi5xImkTgbEHFdVzwJTS5zcC1zpEieaFEkuB/4G+KdJNidZMe6a5hLvpJYkdfIIQpLUyYCQJHUyICRJnQwISVInA0KS1MmA0JyXZGGS65JsSHJfkj9q94RMt88HZqo+aVwMCM1pSQJcA1xbVUuB1wIvB353F7saEJr1DAjNdScAT1fVJQBV9T3g14D3JPm3ST45NTDJDUl+KsmHgYOSfD3JZe29M5OsT3Jnks+2vh9Ksrb1r02yuPV/JsmnktySZFOSN7fnHtyb5DND3/fWJH+T5KtJrkry8hn7T0XCgJB+DLhjuKOqHgceYCfPbK+q84D/V1VHV9UZSX4M+C3ghKp6I3BuG/pJ4NKqegNwGXDR0MccwiCcfg24Hriw1fL6JEcnORz4IPCWqjoGWAe8b2/8YKmvzv8CSHNI6F69dmf9XU4Arq6qRwCqaur5Bf8cOK1tfxb46NA+11dVJbkLeLiq7gJIcg+whMGiia8D/mpwFowDGCw5Ic0YA0Jz3T3Azw93JHklgxVuH+MHj7JfspPP6Bsmw2Oeaa/PDW1PtfcDvgfcXFXv6vG50kh4iklz3VrgpUnOhO8/gvXjDB51uQk4Osk+SRYxePrelH9Msv/QZ7wzyWHtMw5t/X/NYHVcgDOAv9yNum4Djk/ymvaZL03y2t39cdKeMCA0p9VgtcqfA05PsgH438DTDK5S+ivg74G7gI8BXx3adRWwPsllbfXb3wW+mORO4BNtzL8Dzk6yHng32+cm+tS1Dfgl4PK2/23Aj7zQ3ym9EK7mKknq5BGEJKmTASFJ6mRASJI6GRCSpE4GhCSpkwEhSepkQEiSOhkQkqRO/x/B8DV0RrtiXQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x2323722e160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#查看样本是否均衡\n",
    "sns.countplot(data.Outcome)\n",
    "pyplot.xlabel('Outcome')\n",
    "pyplot.ylabel('Number of occurences')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sklearn里分类任务中交叉验证缺省是采用StratifiedKFold（分层参数），保证交叉验证每类数据是均衡的，所以不用太担心，不过要自己实现交叉验证要实现StratifiedKFold的功能"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 特征工程"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pregnancies                   0\n",
      "Glucose                       5\n",
      "BloodPressure                35\n",
      "SkinThickness               227\n",
      "Insulin                     374\n",
      "BMI                          11\n",
      "DiabetesPedigreeFunction      0\n",
      "Age                           0\n",
      "Outcome                       0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# 原数据用缺失值用0补上了，要先用NaN替换0再用\n",
    "NaN_col = ['Glucose','BloodPressure', 'SkinThickness', 'Insulin', 'BMI']\n",
    "data[NaN_col] = data[NaN_col].replace(0, np.NAN)\n",
    "print(data.isnull().sum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pregnancies                 0\n",
      "Glucose                     0\n",
      "BloodPressure               0\n",
      "SkinThickness               0\n",
      "Insulin                     0\n",
      "BMI                         0\n",
      "DiabetesPedigreeFunction    0\n",
      "Age                         0\n",
      "Outcome                     0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# 用中值填补NaN\n",
    "medians = data.median() #求data所有列的中值\n",
    "data = data.fillna(medians)\n",
    "\n",
    "print(data.isnull().sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则化的Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = data.drop('Outcome', axis=1)\n",
    "y = data['Outcome'].values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据标准化\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# 初始化标准化器\n",
    "ss_x = StandardScaler()\n",
    "\n",
    "#分别对训练数据和测试数据标准化处理\n",
    "x = ss_x.fit_transform(x)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(614, 8)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#将数据分成训练集和测试集\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# 随机采样20%的数据构建测试样本，其余作为训练样本\n",
    "# random_state 随机种子的值随便设，设了这个每次取的随机数时固定的\n",
    "x_train, x_test, y_train, y_test = train_test_split(x, y, random_state=6, test_size = 0.2 )\n",
    "x_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "logistic回归的需要调整超参数有：C（正则系数，一般在log域（取log后的值）均匀设置候选参数）和正则函数penalty（L2/L1） \n",
    "目标函数为：J = sum(logloss(f(xi), yi)) + C* penalty \n",
    "\n",
    "在sklearn框架下，不同学习器的参数调整步骤相同：\n",
    "<br/>设置候选参数集合</br>\n",
    "<br/>调用GridSearchCV</br>\n",
    "<br/>调用fit</br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "class sklearn.model_selection.GridSearchCV(estimator, param_grid, scoring=None, fit_params=None,n_jobs=1, iid=True, refit=True, cv=None, verbose=0, pre_dispatch=‘2*n_jobs’, error_score=’raise’, return_train_score=’warn’)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring=None, verbose=0)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "penaltys = ['l1','l2']\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "tuned_parameters = dict(penalty = penaltys, C = Cs) #参数要求写成字典形式\n",
    "\n",
    "lr_penalty= LogisticRegression()\n",
    "grid_lr= GridSearchCV(lr_penalty, tuned_parameters,cv=5) #GridSearchCV\n",
    "grid_lr.fit(x_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([0.00099659, 0.08678026, 0.00079951, 0.00079098, 0.00100031,\n",
       "        0.00079794, 0.00099277, 0.00099497, 0.00100298, 0.0010087 ,\n",
       "        0.00139914, 0.00099931, 0.00100265, 0.00099659]),\n",
       " 'mean_score_time': array([0.00761046, 0.00059366, 0.00019951, 0.00020733, 0.00019717,\n",
       "        0.        , 0.00059576, 0.00019898, 0.00019937, 0.00078659,\n",
       "        0.00020056, 0.        , 0.00059996, 0.        ]),\n",
       " 'mean_test_score': array([0.6465798 , 0.73615635, 0.70684039, 0.75895765, 0.75895765,\n",
       "        0.75732899, 0.76710098, 0.76710098, 0.76547231, 0.76384365,\n",
       "        0.76384365, 0.76384365, 0.76384365, 0.76384365]),\n",
       " 'mean_train_score': array([0.64658039, 0.74186992, 0.70685084, 0.76059233, 0.76995686,\n",
       "        0.772814  , 0.77362369, 0.77362037, 0.77321553, 0.77402854,\n",
       "        0.77402854, 0.77402854, 0.77402854, 0.77402854]),\n",
       " 'param_C': masked_array(data=[0.001, 0.001, 0.01, 0.01, 0.1, 0.1, 1, 1, 10, 10, 100,\n",
       "                    100, 1000, 1000],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_penalty': masked_array(data=['l1', 'l2', 'l1', 'l2', 'l1', 'l2', 'l1', 'l2', 'l1',\n",
       "                    'l2', 'l1', 'l2', 'l1', 'l2'],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'C': 0.001, 'penalty': 'l1'},\n",
       "  {'C': 0.001, 'penalty': 'l2'},\n",
       "  {'C': 0.01, 'penalty': 'l1'},\n",
       "  {'C': 0.01, 'penalty': 'l2'},\n",
       "  {'C': 0.1, 'penalty': 'l1'},\n",
       "  {'C': 0.1, 'penalty': 'l2'},\n",
       "  {'C': 1, 'penalty': 'l1'},\n",
       "  {'C': 1, 'penalty': 'l2'},\n",
       "  {'C': 10, 'penalty': 'l1'},\n",
       "  {'C': 10, 'penalty': 'l2'},\n",
       "  {'C': 100, 'penalty': 'l1'},\n",
       "  {'C': 100, 'penalty': 'l2'},\n",
       "  {'C': 1000, 'penalty': 'l1'},\n",
       "  {'C': 1000, 'penalty': 'l2'}],\n",
       " 'rank_test_score': array([14, 12, 13,  9,  9, 11,  1,  1,  3,  4,  4,  4,  4,  4]),\n",
       " 'split0_test_score': array([0.64516129, 0.72580645, 0.69354839, 0.74193548, 0.71774194,\n",
       "        0.71774194, 0.73387097, 0.73387097, 0.73387097, 0.73387097,\n",
       "        0.73387097, 0.73387097, 0.73387097, 0.73387097]),\n",
       " 'split0_train_score': array([0.64693878, 0.73673469, 0.71020408, 0.75714286, 0.77755102,\n",
       "        0.78163265, 0.78163265, 0.77755102, 0.77959184, 0.77959184,\n",
       "        0.77959184, 0.77959184, 0.77959184, 0.77959184]),\n",
       " 'split1_test_score': array([0.64516129, 0.70967742, 0.66935484, 0.74193548, 0.76612903,\n",
       "        0.75      , 0.75806452, 0.75      , 0.75      , 0.75      ,\n",
       "        0.75      , 0.75      , 0.75      , 0.75      ]),\n",
       " 'split1_train_score': array([0.64693878, 0.76326531, 0.71632653, 0.77142857, 0.76938776,\n",
       "        0.77959184, 0.7755102 , 0.7755102 , 0.7755102 , 0.7755102 ,\n",
       "        0.7755102 , 0.7755102 , 0.7755102 , 0.7755102 ]),\n",
       " 'split2_test_score': array([0.64754098, 0.81147541, 0.78688525, 0.80327869, 0.76229508,\n",
       "        0.78688525, 0.79508197, 0.79508197, 0.79508197, 0.79508197,\n",
       "        0.79508197, 0.79508197, 0.79508197, 0.79508197]),\n",
       " 'split2_train_score': array([0.64634146, 0.7296748 , 0.68699187, 0.75813008, 0.76829268,\n",
       "        0.76219512, 0.7703252 , 0.77439024, 0.77439024, 0.77439024,\n",
       "        0.77439024, 0.77439024, 0.77439024, 0.77439024]),\n",
       " 'split3_test_score': array([0.64754098, 0.7295082 , 0.67213115, 0.76229508, 0.7704918 ,\n",
       "        0.7704918 , 0.77868852, 0.77868852, 0.77868852, 0.7704918 ,\n",
       "        0.7704918 , 0.7704918 , 0.7704918 , 0.7704918 ]),\n",
       " 'split3_train_score': array([0.64634146, 0.7398374 , 0.71544715, 0.75609756, 0.76829268,\n",
       "        0.77439024, 0.7703252 , 0.76829268, 0.76829268, 0.7703252 ,\n",
       "        0.7703252 , 0.7703252 , 0.7703252 , 0.7703252 ]),\n",
       " 'split4_test_score': array([0.64754098, 0.70491803, 0.71311475, 0.74590164, 0.77868852,\n",
       "        0.76229508, 0.7704918 , 0.77868852, 0.7704918 , 0.7704918 ,\n",
       "        0.7704918 , 0.7704918 , 0.7704918 , 0.7704918 ]),\n",
       " 'split4_train_score': array([0.64634146, 0.7398374 , 0.70528455, 0.7601626 , 0.76626016,\n",
       "        0.76626016, 0.7703252 , 0.77235772, 0.76829268, 0.7703252 ,\n",
       "        0.7703252 , 0.7703252 , 0.7703252 , 0.7703252 ]),\n",
       " 'std_fit_time': array([8.01313459e-06, 1.69060776e-01, 3.99778453e-04, 3.95599482e-04,\n",
       "        6.29566307e-04, 3.98976617e-04, 6.23389717e-04, 4.19887542e-06,\n",
       "        6.31439900e-04, 1.73816138e-05, 1.83269694e-03, 2.01254657e-05,\n",
       "        2.60361822e-05, 4.19237225e-06]),\n",
       " 'std_score_time': array([0.01522093, 0.00079547, 0.00039902, 0.00041466, 0.00039434,\n",
       "        0.        , 0.00048645, 0.00039797, 0.00039873, 0.00074635,\n",
       "        0.00040112, 0.        , 0.00119991, 0.        ]),\n",
       " 'std_test_score': array([0.00116767, 0.03864076, 0.04289227, 0.02331156, 0.02143489,\n",
       "        0.02324307, 0.02059191, 0.02215831, 0.02153935, 0.02077477,\n",
       "        0.02077477, 0.02077477, 0.02077477, 0.02077477]),\n",
       " 'std_train_score': array([0.00029262, 0.01132357, 0.01069356, 0.00558164, 0.00392937,\n",
       "        0.00750874, 0.00447979, 0.00314937, 0.00437657, 0.00348442,\n",
       "        0.00348442, 0.00348442, 0.00348442, 0.00348442])}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# view the complete results (list of named tuples)\n",
    "grid_lr.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7671009771986971\n",
      "{'C': 1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "# examine the best model\n",
    "print(grid_lr.best_score_)\n",
    "print(grid_lr.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "E:\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd8VGX2+PHPyaSHBAIJvYogvUiAKIoNBMtPRRcEu+4uuyKu4lqwK7h+ddUVV90FxbaKgquryyoSxIYN6b2DQEINIYGE1Jk5vz9mAiEkZBJmMinn/XrNK7c89865aO7Jvc+95xFVxRhjjDmZkGAHYIwxpuazZGGMMaZCliyMMcZUyJKFMcaYClmyMMYYUyFLFsYYYypkycIYY0yFLFkYY4ypkCULY4wxFQoNdgD+kpCQoO3btw92GMYYU6ssXbr0gKomVtSuziSL9u3bs2TJkmCHYYwxtYqI7PClnd2GMsYYUyFLFsYYYypkycIYY0yFLFkYY4ypkCULY4wxFbJkYYwxpkKWLIwxxlTIkoUxxpgKWbIwJsBunXsrt869NdhhGHNKLFkYY4ypUJ0p92FMTbVuz+Fgh2DMKbMrC2OMz66d9jPXTvs52GGYILBkYYwxpkKWLIwxxlTIkoUxxpgKWQe3McZn28Of9059HNQ4TtXAt64B4Jdba/dxQPUdi11ZGGOMqVBAk4WIDBeRjSKyRUQmlrH+RRFZ4f1sEpGsEuv+KiJrRWS9iPxdRCSQsRpjjClfwG5DiYgDeBUYCqQBi0VktqquK26jqhNKtL8T6OudPhsYBPTyrv4BOA/4NlDx1gXFbwm/NfytIEdijKlrAnllMQDYoqrbVLUQmAlceZL2Y4APvNMKRALhQAQQBuwLYKzGGGNOIpDJohWQWmI+zbvsBCLSDugAfA2gqj8D3wB7vJ8UVV0fwFhNDTPwrWuOdtwZY4IvkE9DldXHoOW0HQ18pKouABE5HegKtPau/1JEBqvqguO+QGQsMBagbdu2fgna1AwT39/umbD6e8bUCIG8skgD2pSYbw3sLqftaI7dggIYASxU1RxVzQG+AJJLb6Sqr6lqkqomJSYm+ilsY4wxpQUyWSwGOolIBxEJx5MQZpduJCJnAPFAyYIzO4HzRCRURMLwdG7bbShjjAmSgN2GUlWniIwHUgAH8KaqrhWRScASVS1OHGOAmapa8hbVR8CFwGo8t67mqur/AhWrMYHUrmhrsEPwm8dneP9mq+W3B+vKcUD1HUtA3+BW1TnAnFLLHis1/0QZ27mAPwQyNmOqyy0fFnomxgY3DmNOhb3BbYwxpkKWLIwxxlTICglibz6bAFNFAHduLqiibje4XGX8VHC7UJcL3O6jPz3Tbu+6Ej/VfUJbz089sa27+HvcqNsFJ+zHfey7S7bR47+7Saana3HvpMmB/AcDtwvU7f14p93uEss8MYH7+LZu94nLtPS8i9Nd+SCw97fDA3gc1eN0dz6uajiTW7IwNVKE5p2wTN1u1OlEC4vAWYQWFXnmS/4sLEK96yheVnJ9UcllZbQrKrW/ku2KSu+vEC3IRwvz0YICz3zx/pyek6u6oYP30Y2NZ/ar5n/FUyACIYIIEBKChHjm4woVUA7/92OOvTalJd6g8k4c97yKL8u0xI/yXsfyBwEB1TAADmdsD+B3VQ/VMMJinQH/HksWpkZxpqdz+MsvaZ6uhBfBxqT+R0/cuFwB/W4JC0PCwiA0FAl1IKEhSAhIiCLiRnAh4kK0iBAtBC1EQtyeNhGKRIGIIhGRSEQ0EtUAomLJ3LwBRUhIOsvzpqooIur5CZ7zl3hOwsfWFS9zI3h/Fi/HDXJs+dE24vasw43o8W0E71/huI61wfNXdvE6Uc/6o8uK/xov7+TtCIeQMHCEeaYdJabLWh5SRrvyloeEepeFg6PEdFnLfY0hJAxCPHfe5wzpBsCl89eVfWy1iOdYHJwe4O+xZGGCrmj/frLnfUn23LnkLl0KqjhC4UgUtLt6hOckHhp29GQuoaHe6dDjT/Cl2xWvL17nECg4hBRkIgUZSP4BJO8AkrsPcvcjOfsgey8cSefEE6RATCLENoMGHSC2+bFPg+YQ28KzLqYphIYft+US74mpy//NqJ5/UH/T4ttCLuZe2gdVuGTeOs/Vh6k3LFmYoCjat4/slHkcTkkhb9kyUCX89I4kjBtH3PBhfHW7p+bkwIceqnhnzkIoPtHn7PX8PG56H2TvgdwMTkgCEuJNAt4Tfsu+3gTQ7FgCiG3haeMI8/8/RG0g4vkrnlDcbjm2zNQrlixMtSnau5fslBQOp8zzJAggolMnEsbfQdywYUScfvyFdIgoZO7wJoI9x076xYmhOCHkZpz4ZeKABk09J/2GraF1P89Jv2QSaNDcmwTs18CYithviQmooj17OJySQvbcFPJWrAAg4owzSLzrT8QOG0bEaaeVuV3ndoc4vU0OvNTr+BUhoZ4TfoNmEN8e2g703gYqdVsoJgFCHAE+Ot+8PcpzW+rSIMfhD3XlWJ68vitQ+48Dqu9YLFkYvyvatYvDKfM4nDKX/JWrAIjo0oXEu+/yJIgOHU6+gz0r6dg6h30ZkTS79fnjk0B0k6OdlMaY6mPJwvhFYVqa5xbT3BTyV68GIKJbVxInTCBu2MWEt2/v245U4YsHKHSGsHJTPBefeWPggq4mO8I6BjsEv6lLx2Iqx5KFqbLC1NRjCWLNGgAiu3cn8c/3EDdsGOFVGWNkzcew82c2bW+E01U3riDcCmgIqooNJW9qK0sWplIKd+7k8NwUsufOJX+d5xn1yJ49aXrfvcRefDHhbdpUsIeT7fwIzHsUWvQm9Yd0P0VcvVSV1KwjzNmwlAWpP7Elezl5IdsgxEXPN89C3A0IcccSShzhEkekNCTK0YiY0EbEhcUTFx5Po4jGxEVEEx3u8H5CiQ53EFVi+ujyCAfRYQ5CHXUjsZqay5KFqVDh9u0cnpvC4ZQUCtZ7yiFH9upF0/vuI3bYMMJblzlabuV9/zfI3g0j34J/3+affQZYfpGLVWlZfLN1PT/t/pntR1biCt+MhOYCEBHSEs3riBDCGQmtOeLMIteVRb57F7msI4d8z44UKPR+ckBdEairAepsgNsVgzpjUWcM6opFnQ1QZwxul2cZ7ijCHY6jiaPMpOKdjipneXR4KFHhDmIiHESHhXr34SAqzEFIiF0NGUsWdcrol9d6JvxQ7qZg269kz/PcYirYsAGAqN69afrAA8RdPJSwVn5KEMUO/go/vQy9roW2yRRIlH/37weqSurBPJbtzOTn7dtZtHcRewtXExK9mZDwLAAiohvTLe4sBrcZxJVnDKZlbDPvWOIuPh71txP2me/M52D+QTLyMjw/8z0/D+QeYH/uAQ7kHeRgfgaZBbvJLjyElvE2dQihRIY0JFziCNU4RGPJdzcgz9mA9IIGFGXHUFgQTX5BNHl5kRS6KnfyjwrzJpQIBxk5VyEhTq57fWGZCak4ycSEH0s4xyerY+siQkPstlwtYsnCHFWwbRuH584le24KBZs2ARDVty/NHpxI7MUXE9aiReC+fN4jnsdihzwJwDPXtQc84+sGy5ECJ6vSDrFsZyZLdu5lRfoyckPW44jZgiNyL0RDVEw0XRqeyUXtB3FR+3NoF9euUifAyNBIWjZoScsGLSts63Q7ySrIIiMv42hSOZpkSiSbjLzNZOQfpEiKIAwokXcjEZpFNCI+ojENw+OJDYsnJrQh0Y5GRITEEUFDQmmIQxsgrlgKnaHkFbk4UuAkr9DF55vXo+5QCp1usnLzyC10klvoIq/QRW6RC5fb97pOIcJxSSUqPJSYEgmn9BVRjI9XR9HhDsLstpzfWbKo5wq2bPH0QaTMpWDzFhAh6swzafbQg54E0bx54IPY8hVs+AwuehziApiQTkJV+fXAEZbvzGLZzkyW7cxg86F1hERvxhGzhdConZDoJkpC6Rrfm/PbjmJQq7Po2rgrjmp6nyM0JJSEqAQSohIqbKuqZBdlczDv4HGJJSM/47hlO3I2kpGfwZGiI2XuJzo0miZRTWgS2YTGDRsTlbcUAfp0Kfvy1eVWnG7F6VKcbrf3p+J0uSlyKy7vT6fLfUK7QreS53JT5FKcuYozx7u9t21lOEIER4gQ5gghNEQIdQihISFHf2bnhiCiXPHBPZXab02UkycIga8uYMminlFVCjZvJntuCofnpVC4ZasnQfQ7k2YPP+xJEM2aVl9AriKYOxHiO8BZd1Tb12bnF7Ey9RDLd2Z6kkNqJtnOXThiNhMZu5WQhtuIalSAIHSO78KgVreQ3CKZvk37EhkaWW1xVpWIEBceR1x4HO0btq+wffHtsBOvUo4lmJ3ZO3GRA8C87fP8F2wIx42s4/B+Iko1UxT1FrhVb3XakvPqrX7rmdejy51AUYl5dUFIpBuAX/PS/HccQSIRihZV/AfEqbJkUQ+oKgWbNh19zLVw2zYQITopifhHxxA7dChhTasxQZS06DU4sAnGzILQ0qcH/3C7lW0Hcli2I4vlqZks25HFpv3Z4MjCEbOF+CY7CGu/mRj19Du0iW3LWS2uJLllMv2b9adRZKOAxFWT+Ho7zNP/AgtGf1wdYQVM8XH8cmvtPg7wHktgfnWOE9BkISLDgZfw/KEwXVWfKbX+ReAC72w00FRVG3nXtQWmA23w/LFwqapuD2S8dYmqUrBx49E+iMLt2yEkhOj+/Ym/4Xrihg4lNDExuEHmpMO3z8DpQ6DzML/t9lBuEctTM1m+M4vlqVms2JnJ4XwnhOQS22gHCYk7adV0E4ecuwCIjmzMwBZncVaLsxjYYqBP/QfG1DcBSxYi4gBeBYYCacBiEZmtqkcLyKvqhBLt7wT6ltjFv4C/qOqXItIATzF+czKqhBW62f+3F8lOSaFwxw5Pghg4gMa33EzskCGEJgT+ctVnXz0JRbkw/JkqVzF1uZXN+7M9Vw3eW0pb0z3330NCimjXMp2OZ2wn17GBvXlbcOPmSGgUSc2SSG5xHQNbDKRTfCdCxDpEjTmZQF5ZDAC2qOo2ABGZCVwJlDfayBjgcW/bbkCoqn4JoKo5AYyzTjiyaBFNd+cRme8i4403iBk4gMa33Ubs0CGENm4c7PBOtGspLH8Pzh4PCZ183uzgkUJWeG8lLU/NZGXqIXIKPKOExceEckabw3To+CuHZT1bs1dzwFVAZpGDXo16cUXnsSS3SKZXQi/C6mu5cWOqKJDJohWQWmI+DRhYVkMRaQd0AL72LuoMZInIf7zL5wMTVdVVaruxwFiAtlUpLVEH5C5dSvrLr5C7cCGhDiGzSQQDPvuG0Pj4YIdWPrcbvnjAUx588P3lNlMV1uw6xPLULJbvyGR5aha/HvBcNThChC4tGjC0l4Oohjs46FrLmoNLWVt4CA7B6Y1OZ1TnkSS3SKZfs340CG9QXUdnTJ0UyGRR1n2F8p5/Gw18VCIZhALn4rkttROYBdwCvHHczlRfA14DSEpKCuTAvTVO7vLlHHj5FY789BOOJk1ImHgfLy37Oz93chO34Daiw6KJCo06+okOPTZfvC46NJqosFLrSrULCwnz/4tTqz+EtMVw5T8gMq7MJkcO9iDnQF8u3/QDAAkNIjizbSMu7xtLeMxW9hatZsm+RczP2Q050Cy6Gee3OZ/klskMbD6QxOgg98cYU8cEMlmk4emcLtYa2F1O29FAyecm04DlJW5hfQokUypZ1Ed5q1eT/vLLHFnwPY74eJredx9pQ7vz4Iq/sqmfiw7pIbSNbUueM49cZy6Z+Zme6aJc8px55DnzynwLuDwOcRyXWEonlJLJ57h1YeW0c7uImv8YUa3OJLT3mDK/c+G2DHLS+xMevYunLruI0OjtbM1ZzsI9C3k7zfOyYGxYLANaDODWHreS3CK50i/DGWMqJ5DJYjHQSUQ6ALvwJITrSjcSkTOAeODnUtvGi0iiqqYDFwJLAhhrjZe/bh3pL79Czjff4GjYkMQ/34PjN5fz8obX+eibKSRGJ3Lzj+H02BXC8HtfKnc/qkqBq4Bcpzd5FOUdm/Z+SiaW49aVaHu48DB7j+w9brt8V75vB5MQARwgfEb/E5JKWEgkq1PziGqthDXYwf+tm4rT7SQsJIwzm57JXWfeRXKL5Gp9Gc4YE8BkoapOERkPpOB5dPZNVV0rIpOAJao629t0DDBTVbXEti4RuRf4Sjx/Li4FXg9UrDVZ/saNHHjlFbK/nE9IXByJd99Fo+uv54v93/L8vNEcKjjEjd1uZFyfcfzw9gUV7k9EiAyNDMiLZS63i3xXftkJpyiP3Mxfyft6Enmt+pLbZXiZyWr17nSKyCE05iCCm5u63VSrXoYzpq4K6HsWqjoHmFNq2WOl5p8oZ9svgV5lrasP8jdt4sCr/yA7JYWQBg1IGD+exjffxE73AR746W4W7V1Ez4SeTBs6jS6NuwQ7XAAcIQ5iQmKICYs5rh7RUT+Mgnw3XPqaZwzsUj5ZnsYX81dy78WdeXfHRFCY0G9CGTsyxlQ3e4O7hinYto0Dr7zK4S++ICQ6moRxt9P45pspiongn6un8+aaN4l0RPJo8qNc0+ma2nMrZlMKbE6Bi58qM1GkHszl0U/X0r99PLeffzrvvhOEGI0x5bJkUUMUbt9O+j/+weHPPkciI2ny+9/T+NZbCI2P56ddP/HUV0+Rmp3KZaddxr1J95ZZTG7mnd0B8N+70H7iLIC5D0KTTjDgDyeudrmZMGsFArx4bR8cNn6CMTWOJYsgK9y5kwP/+CeHZs9GwsNpfOstNPntbwlt3Jj03HT++t19zN0+l3Zx7Xj94tdJbpEc7JArb+E/4eBWuP5jCA0/YfU/v93Kkh2ZTLm2D63jo4MQoDGmIpYsgqQwbRcHpv6TQ598ioSG0vjGG2ny+98RmpCAy+3igw0f8Pdlf6fQVci4PuO4rcdtRDiqoVqYv2XvhQXPQedLoNOQE1Yv35nJlK82c0XvllzV188DKhlj/MaSBf4dYa4iRbt3c2Daa2R9/DESEkL8ddfR5Pe/O1r1dV3GOib9PIm1GWtJbpHMI8mP0C6uXeADC5T5T4CrEIb95YRVRwqcTJi1guZxkUy+qkf1x2aM8Zkli2pStG8fGdOmkfXvj1AgftRImowde3RwoZzCHF5Z8QofbPiA+Ih4nj33WS7pcEntftEsdRGs/ADOuQeadDxh9aT/rWPHwVxm/j6ZhlFWq8mYmsySRYAV7d9PxuvTyZo1C3W7aXT11ST88Q+EtfSUwVZVvtzxJc8uepb0vHRGnTGKP535J+LCyy6DUWu43TDnPohtAef++YTVc9fsYdaSVMad35GBpzUJQoDGmMqwZBEgzgMHyHh9OpkzZ6JOJw1HXEXCH/9IeOvWR9ukZqfy9C9P88OuH+jSuAtTLphCz8SeQYzaj1a8B3tWwNWvQ8TxRfz2Hspn4n9W06t1Q+4e0jlIARpjKsOShZ85Dx4k4403yJzxPlpYSMMrriBh3O2El6iKW+Qq4u21bzNt1TQc4uD+/vczpssYQkPqyH+OvCyY/yS0GQg9Rx63yu1W7v33SgqK3Ey5tg/hoTaOhDG1QR05OwWfMzOTg2+9zcH33kPz8oi7/HISxt1ORIcOx7VbvHcxTy18im2HtjG03VDu738/zWOa+yWGdXsO+2U/p+y7v0JuBtzw8QmDGr3546/8sOUAT4/oyWmJVjbcmNrCksUpch06xMF33uHgO//CnZtL3CWXkHDHOCI6Ht+hezD/IC8seYHZW2fTqkErXr3oVQa3HhykqANo/wZYNA363Qwt+xy3at3uw/x17kaGdmvGmAFtytmBMaYmsmRRRa7sbA6+8y8OvvMO7uxsYocNI+GOcUR2Pv4evFvdfLL5E/629G/kFuXyu56/Y2yvsUSFllU8qZZThbkTISwGLnz0uFX5RS7unrWchtFhPHtNr9r9lJcx9ZAli0py5Rwh8713yXjzLdyHD9NgyEUkjh9PZJcTi/ltytzEUwufYvn+5ZzZ9EweTX6U0+NPD0LU1WTjHNj2DQx/FmKOL0fyzBcb2LQvh3duG0DjmBPf4jbG1GyWLHzkPnKEg++/z8E33sSVlUWD888nYfx4onp0P6FtblEuU1dN5d2179IgvAGTB03myo5X1u2/povyPfWfErtC/98et+rbjft5+6ft3DqoPed1rn8j2HVrUcsfgzYGSxYVcuflkfn+B2S88QaugweJGXwuiePHE9Wr7Orp36Z+y9O/PM2eI3sYcfoI7ul3D40iG1Vz1EHw88uQtQNu+i84jr1gl5FTwL3/XsUZzWJ5YHjNKKVujKk8SxblcOfnkzVrFgden47rwAFizj6bhDvHE923b5nt9x7ZyzOLnuGrnV/RsWFH3h7+Nv2a9avmqIPk0C74/m/Q9f/BaecfXayqPPDxag7nFfHubwcQGVZLyqn72VvD3wp2CMacMksWpbgLCsj690dkTJuGMz2d6IEDSZzyItFJSWW2d7qdzFg/g1dXvIqqcveZd3NTt5sIc9Sj8hVfPgbqhouPr//0/qKdzF+/j0cv70ZXuxVjTK1myaKYKpkffMCBaa/h3LuXqKR+tHz+eWIGDih3k5XpK5n882Q2Zm5kcOvBPDTwIVo1qGeVU3f8BGs+gvMegPhjBQ+3pucw+bN1nNspgVvPbh+8+IxfWf9L/RXQZCEiw4GX8IzBPV1Vnym1/kWgeODoaKCpqjYqsT4OWA98oqrjAxGjFhURc7iQhpmF7H1yElF9+9Ly/54mOjm53A7pQwWHeGnZS3y06SMSoxN58fwXuajtRXW7A7ssbhfMuR/iWsOgu48uLnS6uXvmCqLCHDw/sjchNphRnVFXbqn9cuvHwQ7Bb6rrWAKWLETEAbwKDAXSgMUiMltV1xW3UdUJJdrfCZTuEJgMfBeoGAGK9u2ncXoBhREhdPjna8ScM6jck76q8tm2z3h+yfNkFWRxQ7cbuKPPHZ4xp+ujpW/DvtXwm7cg/NigRS/O38TqXYeYekM/msVFBi8+Y4zfBPLKYgCwRVW3AYjITOBKYF057ccAjxfPiEg/oBkwFyi7w8APwlu3Ym/raIrCQ+hz7jnltvv10K88tfApFu1dRM+EnkwdMpWuTboGKqyaL/cgfP0UtDsHuo84unjhtgymfreV0f3bMLyHf8qYGGOCL5DJohWQWmI+DRhYVkMRaQd0AL72zocALwA3AhcFMEYAiiLKf0qnwFXA9NXTeWP1G0Q6Ink0+VGu6XQNjpD6+WTPUd/+H+RnwSXPHq3/dCiviHtmraBd42gevbxbkAM0xvhTIJNFWfdytJy2o4GPVNXlnR8HzFHV1JP1A4jIWGAsQNsSVV395addP/HUL0+Rmp3KZaddxr1J95IQlVDxhnXdvrWweDok/Raae0a4U1Ue+XQN+7ML+Pj2s4mJsGcnjKlLAvkbnQaUrBbXGthdTtvRwB0l5s8CzhWRcUADIFxEclR1YsmNVPU14DWApKSk8hJRpaXnpvPc4uf4YvsXtItrx+sXv05yi2R/7b52U4UvHoDIhnDBQ0cXf7piF/9buZt7L+5M7zb14CVEY+qZQCaLxUAnEekA7MKTEK4r3UhEzgDigZ+Ll6nq9SXW3wIklU4UgeByu/hw04f8fdnfKXAVMK73OG7reRsRjohAf3Xtse5T2P49XPYCRDcGIPVgLo99upb+7eO5/fw6XPvKmHosYMlCVZ0iMh5IwfPo7JuqulZEJgFLVHW2t+kYYKaq+u3KoCrSGrm5Yc4NrMlYQ3KLZB5JfoR2ce0q3rA+KcyFeY9Cs57Q71YAnC43E2atAOBvo/rg8NNjsvY8vzE1S0BvLKvqHGBOqWWPlZp/ooJ9vA287efQjsopzOG/fQr54XQXjY/s4dlzn+WSDpfUv3cmfPHjS3AoFUZMA28H/z+/3cqSHZlMubYPbRpHV7ADY0xtVe97IXOduSxp7+KsbQ6ee3Q2ceH2F22ZMnfAj1Og+9XQfhAAK1KzmPLVZq7o3ZKr+tazN9eNqWfqfbJoGt2UB+dEEl0otT5RtC+8N3A7n/cIIHDxZACOFDi5e+ZymsdFMvmqHoH7XmNMjRAS7ABqguhCu+V0Utu+g/Wz4dw/Q8PWAEz63zp2HMzlb6N60zCqHhVNNKaeqvdXFqYCLqfnUdlG7eDsOwGYu2YPs5akMu78jgw8rUlAvrau1CAypq6wZGFObskbkL4ern0PwiLZeyifif9ZTa/WDbl7SOeKtzfG1Al2G8qU78gB+OYvngGNulyO263c+++VFBS5mXJtH8JD7X8fY+oLn37bReRyb70mU598/RQU5MBwT/2nN3/8lR+2HODRy7txWmKDYEdnjKlGviaA0cBmEfmriNTjUqv1yJ6VnhLkA/8ATbuwbvdh/jp3I0O7NWPMgDYVbm6MqVt86rNQ1Ru8AxGNAd4SEQXeAj5Q1exABlgdZt7ZHYBhQY6jxlD1DGoU3QTOe4D8Ihd3z1pOw+gwnrm6p72waEw95POtJVU9DHwMzARaACOAZd5Bi0xdsvojSF0IFz0GUY145osNbNqXw/Mje9OkgdXJMqY+8rXP4v+JyCd4xpsIAwao6iVAbyCAb4KZaleQA18+Bi36QN8b+Hbjft7+aTu3nN2e8zonBjs6Y0yQ+Pro7EjgRVVdUHKhquaKyG3+D8sEzQ9/g+zdMPJtMnKd3PvvVZzRLJaJl3QJdmTGmCDy9TbU48Ci4hkRiRKR9gCq+pX/wzJBcXAb/PQy9LoWbTOABz5ezeG8IqaM7kNkWD0fGdCYes7XZPFvwF1i3uVdZuqSlIchJAyGPMn7i3Yyf/0+7h9+Bl2tXLgx9Z6vySJUVQuLZ7zT4YEJyQTFlvmwcQ6cdx9bC2KZ/Nk6zu2UwG2DOgQ7MmNMDeBrskgXkSuKZ0TkSuBAYEIy1c5ZCF9MhManUZj0R+6euYKoMAfPj+xNiJ8GMzLG1G6+dnD/EZghIq8AAqQCNwUsKlO9Fr0GGZthzCymfLuD1bsOMfWGfjSLiwx2ZMaYGsLXl/K2Aski0gCQuvAinvHK3gffPgOnD2VhaBL//O4XRvdvw/AezYMdmTGmBvG56qyIXAZ0ByKL3+BV1UkBissFOJh/AAAfRklEQVRUl68mgTOf7PMncc+7K2nXOJpHL+8W7KiMMTWMry/lTQWuBe7EcxtqJNDOh+2Gi8hGEdkiIhPLWP+iiKzwfjaJSJZ3eR8R+VlE1orIKhG5tlJHZXyTthRWvIcm385D3xewL7uAKaP7EhNhleuNMcfz9axwtqr2EpFVqvqkiLwA/OdkG4iIA3gVGAqkAYtFZLaqrituo6oTSrS/E+jrnc0FblLVzSLSElgqIimqmuX7oZmTcrvhi/shpimfNbqe/329lXsv7kyfNo2CHZkxpgby9WmofO/PXO/Juwio6JnKAcAWVd3mfdR2JnDlSdqPAT4AUNVNqrrZO70b2A9YrQl/WjUTdi0h4+yHeejzHfRvH8/t558e7KiMMTWUr8nifyLSCHgOWAZsx3tiP4lWeJ6aKpbmXXYCEWmHJ/l8Xca6AXje6dhaxrqxIrJERJakp6f7cBgGgPzD8OXjaKsk/rjSkyD+NqoPDntM1hhTjgqThXfQo69UNUtVP8bTV9FFVR+raNMylmk5bUcDH6mqq9R3twDeBW5VVXfpjVT1NVVNUtWkxES78PDZgufgyH4+TLyTxTsPMfmqHrRpHB3sqIwxNViFycJ7kn6hxHyBqh7yYd9pQMlRcloDu8tpO5pSVyre8TM+Bx5R1YU+fJ/xxYHNsPCfZHQayUOLwrmid0uu6lvmBZ8xxhzl622oeSJyjVRu1JvFQCcR6SAi4XgSwuzSjUTkDCAe+LnEsnDgE+Bfqmo1qPxFFeZORMMi+d2uy2geF8nkq3oEOypjTC3g69NQ9wAxgFNE8vHcYlJVLbfCnKo6RWQ8kAI4gDdVda2ITAKWqGpx4hgDzFTVkreoRgGDgSYicot32S2qusLXA6uPHsu4zzv1Q9kNNqXAlvl83vwOVuwIZ+bve9MwKqza4jPG1F4VJgvv1UR3Vd1Z2Z2r6hxgTqllj5Waf6KM7d4D3qvs95mTcBZAyoPkxHZgwvaB3H5+Rwae1iTYURljaglf+iwUzy0hU5st/Acc3Mb9OWPo0qoJdw/pHOyIjDG1iK99FgtFpH9AIzGBc3gP+t1zLI08i2+cvZkyug/hoT4Pv26MMT73WVwA/EFEdgBHONZn0StgkRn/mf84bmchE3JG8ehV3eiY2CDYERljahlfk8UlAY3CBM7OX2DVLF53XckZXXsxZkCbircxxphSfE0W5b1MZ2oytwv3nPs5KI15P2Ikn1zdk8o9/WyMMR6+JovP8SQMASLxlObYiKdkuamplr9HyN4VTCq8g8m3JNOkQUSwIzLG1FK+Dn7Us+S8iJwJ/CEgERn/yMuicN7jrHR3pvHA6zivs5VDMcZUXZUeiVHVZYA9HVWD5c1/mtCCLN6Mu52Jl3YNdjjGmFrOpysLEbmnxGwIcCZgZV5rKN2/nvClr/Oh60L+dP1viAxzBDskY0wt52ufRWyJaSeePoyP/R+OOWWq7Jt1F1EaifP8h+jaotyKLMYY4zNf+yyeDHQgxj8KXUrzjF94J34cN17QL9jhGGPqCF/H4P7SO/hR8Xy8iKQELixTFU43NHftYQttGH7Lw4TYYEbGGD/xtYM7seT416qaCTQNTEimqjKKwmkhBzl47mSaNbK3tI0x/uNrsnCJSNviGRFpj72oV6Ps3rqWQaziZ3c3Blw0ItjhGGPqGF87uB8GfhCR77zzg4GxgQnJVJa63WT8+07icEBYVLDDMcbUQT5dWajqXCAJz1vbs4A/A3kBjMtUwoqUt+mZv5Qf6U2cozDY4Rhj6iBf37P4HXAXnnG0VwDJeIZBvTBwoRlf5Bw+SOtfJrHVcRotJDfY4Rhj6ihf+yzuwvPG9g5VvQDoi72UVyOsnTGRJppF4SUv4LAhKowxAeLr6SVfVfMBRCRCVTcAZ1S0kYgMF5GNIrJFRCaWsf5FEVnh/WwSkawS624Wkc3ez82+HlB9sm31zyTt/ZDFTa6ga5Jd5BljAsfXDu4073sWnwJfikgmsPtkG4iIA3gVGAqkAYtFZLaqrituo6oTSrS/E88VCyLSGHgcTz+JAku922b6fGR1nNvlomj23RySWLrc8HywwzHG1HG+dnCPUNUsVX0CeBR4A7iqgs0GAFtUdZuqFgIzgStP0n4M8IF3ehjwpaoe9CaIL4HhvsRaXyz9dApnFG1gS58HaNjYXnkxxgSWr1cWR6nqdxW3AqAVkFpiPg0YWFZDEWmHZ4yMr0+ybavKRVp3ZabvpvPqF1gX1pP+V4wLdjjGmHogkF2iZdWaKO9FvtHAR6rqqsy2IjJWRJaIyJL09PrT3751xj1Eaz5RI6YgIdarbYwJvECeadKAkgM+t6b8fo7RHLsF5fO2qvqaqiapalJiYv0Y3GfDL3NJyvqCJS2vo0O3pGCHY4ypJwKZLBYDnUSkg4iE40kIs0s3EpEzgHg8720USwEu9hYsjAcu9i6r14oKC4hMuY89JNLrur8EOxxjTD1S6T4LX6mqU0TG4znJO4A3VXWtiEwClqhqceIYA8xUVS2x7UERmYwn4QBMUtWDgYr1reFvBWrXfrVs1l8Y6N7JskH/oEVsw2CHY4ypRwKWLABUdQ4wp9Syx0rNP1HOtm8CbwYsuFpm387N9NwylRXRZ9F3yHXBDscYU88ENFkY/9kz6y7iUJpe+xIiZY9TManJc4CneJcxxviTPUpTC6z6aiZ9jvzIitPG0rJ9hS/OG2OM31myqOHyc7NJ/OFRtoe0od/oR4MdjjGmnrJkUcOtnPEILXQ/OUOeJTwiMtjhGGPqKUsWNdjOjcvpm/YuixteTI+zLwt2OMaYesySRQ2lbjfZH/+JfImkw3UvBjscY0w9Z8mihlr2+TS6F65iXbcJJDRrHexwjDH1nCWLGuhwVjrtl/4fG0M70//qCRVvYIwxAWbJogba8N79NNLDhFz+Io5QexXGGBN8lixqmC3LF5CU/gmLm15Dpz7nBDscY4wBLFnUKC6nE/18AgelId2u/2uwwzHGmKMsWdQgSz96jk7OLWxPeoS4Rk2CHY4xxhxlN8RriAN7dtB1w0usjuhLv0t/G+xwjKlVioqKSEtLIz8/P9ih1FiRkZG0bt2asLCwKm1vyaKG2PHBPfTUIhr+5iUb/c6YSkpLSyM2Npb27duXW2izPlNVMjIySEtLo0OHDlXah52VaoC1P/yXfofns7TNzbTt1DvY4RhT6+Tn59OkSRNLFOUQEZo0aXJKV16WLIKsMD+PuK8mskua0fe6ScEOx5haq7KJ4tppP3PttJ8rblhHnGoitWQRZMtmPkkb3c2BwU8TGd0g2OEYY6qoQYNjv7/Dhw+nUaNGXH755WW2veOOO+jTpw/dunUjKiqKPn360KdPHz766KNKfeeyZcuYO3fuKcXtK+uzCKLdv66nz6/TWdpgMP0u+E2wwzHG+Ml9991Hbm4u06ZNK3P9q6++CsD27du5/PLLWbFiRZW+Z9myZaxZs4bhw4dXOVZf2ZVFkKjbTfqHf8KFg1ajpwQ7HGOMH1100UXExsZWadvNmzczbNgw+vXrx+DBg9m0aRMAM2fOpEePHvTu3ZsLLriAvLw8Jk2axIwZM6p0VVJZAb2yEJHhwEuAA5iuqs+U0WYU8ASgwEpVvc67/K/AZXgS2pfAXaqqgYy3Oq348j365i1iYad7SG7TMdjhGFNnPPm/tazbfbjCduv2eNr40m/RrWUcj/+/7qccmy/Gjh3L9OnT6dixIz/++CPjx49n3rx5PPnkk3z77bc0a9aMrKwsoqKieOyxx1izZg1TpgT+D86AJQsRcQCvAkOBNGCxiMxW1XUl2nQCHgQGqWqmiDT1Lj8bGAT08jb9ATgP+DZQ8VanI9lZtPz5CbaFtKfftQ8GOxxjTA2RlZXFwoULueaaa44uczqdAAwaNIibbrqJkSNHcvXVV1d7bIG8shgAbFHVbQAiMhO4ElhXos3vgVdVNRNAVfd7lysQCYQDAoQB+wIYa7VaM+NBBpLB+uHTCAsLD3Y4xtQpvl4BFF9RzPrDWYEMp1JUlYSEhDL7MF5//XV++eUXPvvsM3r37s2qVauqNbZA9lm0AlJLzKd5l5XUGegsIj+KyELvbStU9WfgG2CP95OiqutLf4GIjBWRJSKyJD09PSAH4W+/rl1Evz0zWRR/OV0HDA12OMaYGiQ+Pp4WLVrwySefAOB2u1m5ciUA27ZtIzk5mcmTJxMfH8+uXbuIjY0lOzu7WmILZLIo66He0n0OoUAn4HxgDDBdRBqJyOlAV6A1ngRzoYgMPmFnqq+papKqJiUmJvo1+EBwu1wUfHoX2RJD5+tfCHY4xpgAOffccxk5ciRfffUVrVu3JiUlxedtZ86cydSpU+nduzfdu3fns88+A2DChAn07NmTnj17MmTIEHr06MGFF17IypUr6du3b63u4E4D2pSYbw3sLqPNQlUtAn4VkY0cSx4LVTUHQES+AJKBBQGMN+CW/vcV+hetY1GvSQxIaB7scIwxfpSTk3N0+vvvv/dpm/bt27NmzZrjlp122mllJpfZs2efsCwxMZElS5ZUMtKqCeSVxWKgk4h0EJFwYDRQ+mg/BS4AEJEEPLeltgE7gfNEJFREwvB0bp9wG6o2yTqwl9NXPcf6sG4kXTk+2OEYU+/N+sNZNaq/oqYLWLJQVScwHkjBc6L/UFXXisgkEbnC2ywFyBCRdXj6KO5T1QzgI2ArsBpYieeR2v8FKtbqsHnGPcTqESKunEKIwxHscIwxplIC+p6Fqs4B5pRa9liJaQXu8X5KtnEBfwhkbNVpw6L59M/8nIXNx5DcY2CwwzHGmEqzN7gDzFlUSHjKn9lHE3recMI7icYYUytYsgiwJR/+H6e5trP7rMeJiW0U7HCMMaZKLFkE0P60bfTc9A9WRg2gz9Abgx2OMaakty7zfIxPLFkE0K6Zd+HARcLIv9vod8bUcdVdovyTTz7hueeeO+W4fWUlygNk9bcf0TdnAT+3/yNnndY12OEYY6qRv0qUO51OQkPLPk2PGDHCP8H6yP7cDYD83Bwaf/cQO0NaceaYx4MdjjGmmp1KifJzzjmHhx9+mMGDB/PKK6/w3//+l4EDB9K3b18uvvhi9u/3lNCbPn06d999NwA33HADd911F2effTannXba0XIh/mRXFgGw4v3HSNZ9rLnoXdpGRgc7HGPqly8mwt7VFbfb6y3E50u/RfOecEn1Pc14+PBhFizwFKzIzMzkiiuuQESYOnUqL7zwAs8+++wJ2+zfv58ff/yR1atXM2rUKL9feViy8LPUzas4M/UdlsQNIemcKyrewBhjShk9evTR6Z07dzJq1Cj27t1LQUEBnTt3LnObq666ChGhV69e7Nq1y+8xWbLwI3W7OfTRn2hEGO2vfzHY4RhTP/l6BVB8RXHr54GLpYpiYmKOTt9xxx089NBDXHrppcyfP59nnin7+CIiIo5OB2KcOOuz8KNlX7xBj4LlrOt6FwnN2wY7HGNMHXDo0CFatWqFqvLOO+8ELQ5LFn6SnZVBu8VPsdlxOkm/uS/Y4RhjguhUSpSX9sQTTzBixAjOO+88mjVr5scoK8duQ/nJuhn3018PkXn5uzjKedTNGFN3+atE+Q8//HDc/DXXXHPcMKvFfve73x2dfu+998qNxV/srOYHW1b+QNL+j1mcOIKBfU8Yo8kYUxPVwL6KmsxuQ50it9OJ+38TyJI4ulxffW9TGmNMdbJkcYqWfPIinZ2b+PXMB2kYnxDscIwxJiAsWZyCjH2pdFn7ImvDe9Pv8joz/IYxxpzA+ixOwbb376G35tPgmppRKNCGiDTGBErwz3C11LqfPqf/oXksbX0j7c7oE+xwjDGVdOvcW7l17q3BDqPWCGiyEJHhIrJRRLaIyMRy2owSkXUislZE3i+xvK2IzBOR9d717QMZa2UUFuQTM/8BdktT+lz3VLDDMcbUAMUlylesWMFZZ51F9+7d6dWrF7NmzTqhrT9KlAMsW7aMuXPn+iX+igTsNpSIOIBXgaFAGrBYRGar6roSbToBDwKDVDVTRJqW2MW/gL+o6pci0gBwByrWylo28ymS3amsOHcaLWOqVlnSGFM3RUdH869//YtOnTqxe/du+vXrx7Bhw2jU6NhImb6WKK/IsmXLWLNmDcOHD/dL7CcTyCuLAcAWVd2mqoXATODKUm1+D7yqqpkAqrofQES6AaGq+qV3eY6q5gYwVp/t2bGR3tumsTx6EH0uGl3xBsaYeqVz58506tQJgJYtW9K0aVPS09N93n7z5s0MGzaMfv36MXjwYDZt2gTAzJkz6dGjB7179+aCCy4gLy+PSZMmMWPGjCpdlVRWIDu4WwGpJebTgIGl2nQGEJEfAQfwhKrO9S7PEpH/AB2A+cBEVXUFMF6f7Jt1Fw0RWox+KdihGGPK8OyiZ9lwcEOF7Yrb+NJv0aVxFx4Y8EClY1m0aBGFhYV07NjR523Gjh3L9OnT6dixIz/++CPjx49n3rx5PPnkk3z77bc0a9aMrKwsoqKieOyxx1izZg1TpkypdGyVFchkIWUsK10KMRToBJwPtAa+F5Ee3uXnAn2BncAs4BbgjeO+QGQsMBagbdvAF+5b8eUM+uT+zMLT7yK5baeAf58xpvbas2cPN954I++88w4hPj4tmZWVxcKFC48r7+F0OgEYNGgQN910EyNHjuTqq68OSMwnE8hkkQa0KTHfGthdRpuFqloE/CoiG/EkjzRguapuAxCRT4FkSiULVX0NeA0gKSnJ/zV5S8jNOUTzHx9ne0hb+l37cCC/yhhzCny9Aii+onhr+Ft+j+Hw4cNcdtllPPXUUyQnJ/u8naqSkJBQZh/G66+/zi+//MJnn31G7969WbVqlT9DrlAg+ywWA51EpIOIhAOjgdml2nwKXAAgIgl4bj9t824bLyKJ3nYXAusIopUzHqE56eQNe46w8IiKNzDG1EuFhYWMGDHi6FVAZcTHx9OiRYujw6K63W5WrlwJwLZt20hOTmby5MnEx8eza9cuYmNjyc7O9vsxlCVgyUJVncB4IAVYD3yoqmtFZJKIFA8hlwJkiMg64BvgPlXN8PZN3At8JSKr8dzSej1QsVZkx/olJO2eweJGl9B1YOCfOjDG1F4ffvghCxYs4O233z76SGxlnnaaOXMmU6dOpXfv3nTv3p3PPvsMgAkTJtCzZ0969uzJkCFD6NGjBxdeeCErV66kb9++Ae/glkCMqBQMSUlJumTJEr/vV91u1j8zmFaF23CNW0zjpq38/h3GmFOzfv16unbtWqltAnkbqqYq699JRJaqalJF21q5jwosmf1P+heuZlGPxxlgicKYOqM+JQl/sHIfJ3EoYx8dVzzDxtAuJI24K9jhGGNM0FiyOIkN799HQ80m9IophDgcwQ7HGGOCxpJFOTYu+Zr+B2azuNkoOvayaq7GmPrNkkUZnEWFhH7xZw5IPD1ueDbY4RhjTNBZsijDko+eo6NrG6kDHqNBXHywwzHGBMCOG29ix403BTuMWsOSRSnpu7fTY8PLrI7sx5nDbw52OMaYWqK6S5R/8sknPPfcc36LvyL26GwpOz+4mx44if9NzRj9zhhTu/izRLnT6SQ0tOzT9IgRI/wf/ElYsihh9YJP6Jf9DQvbjSX59B7BDscYUwt17tz56HTJEuUlk8XJnHPOOZx33nl8//33XH311XTo0IGnn36awsJCEhMTee+992jatCnTp08/WnH2hhtuoEmTJixevJi9e/fywgsv+D2ZWLLwys87Qvw3D5ImLegz5olgh2OMqaK9Tz9NwfqKS5Tnb/C08aXfIqJrF5o/9FClY6lKiXLwFCJcsGABAJmZmVxxxRWICFOnTuWFF17g2WdPfPBm//79/Pjjj6xevZpRo0ZZsgiU5R88yVm6h9UXvkXrqJhgh2OMqeWqUqK82OjRxwZW27lzJ6NGjWLv3r0UFBQcd+VS0lVXXYWI0KtXL3bt2nVKsZfFkgWQtmUNZ+54k6Wx59NvcPXXiTfG+I+vVwDFVxTt3v2X32OoaonyYjExx/5gveOOO3jooYe49NJLmT9/Ps8880yZ20REHKuGHYiaf/W+B1fdbna8N45CQml7nY1+Z4w5NadSorwshw4dolWrVqgq77zzjh8irJp6nyxSt6yin65jIT1JbNk+2OEYY2q5Uy1RXtoTTzzBiBEjOO+882jWrJkfI60cK1EO/P6l/5AdnsjM28/1c1TGmOpQlRLlgbwNVVNZifJT9Ppd1k9hTH1Tn5KEP9T721DGGGMqZsnCGGNMhSxZGGPqhLrS/xoop/rvE9BkISLDRWSjiGwRkYnltBklIutEZK2IvF9qXZyI7BKRVwIZpzGmdouMjCQjI8MSRjlUlYyMDCIjI6u8j4B1cIuIA3gVGAqkAYtFZLaqrivRphPwIDBIVTNFpGmp3UwGvgtUjMaYuqF169akpaWRnp4e7FBqrMjISFq3bl3l7QP5NNQAYIuqbgMQkZnAlcC6Em1+D7yqqpkAqrq/eIWI9AOaAXOBCh/rMsbUX2FhYXTo0CHYYdRpgbwN1QpILTGf5l1WUmegs4j8KCILRWQ4gIiEAC8A953sC0RkrIgsEZEl9heFMcYETiCThZSxrPQNxVCgE3A+MAaYLiKNgHHAHFVN5SRU9TVVTVLVpMTERD+EbIwxpiyBvA2VBrQpMd8a2F1Gm4WqWgT8KiIb8SSPs4BzRWQc0AAIF5EcVS2zk9wYY0xgBazch4iEApuAi4BdwGLgOlVdW6LNcGCMqt4sIgnAcqCPqmaUaHMLkKSq4yv4vnRgxymEnAAcOIXta4q6chxgx1JT1ZVjqSvHAad2LO1UtcJbMwG7slBVp4iMB1IAB/Cmqq4VkUnAElWd7V13sYisA1zAfSUTRSW/75TuQ4nIEl/qo9R0deU4wI6lpqorx1JXjgOq51gCWhtKVecAc0ote6zEtAL3eD/l7eNt4O3ARGiMMcYX9ga3McaYClmyOOa1YAfgJ3XlOMCOpaaqK8dSV44DquFY6sx4FsYYYwLHriyMMcZUyJKFl4hMFpFVIrJCROaJSMtgx1RVIvKciGzwHs8n3hcdayURGektMukWkVr35IovxTRrCxF5U0T2i8iaYMdyKkSkjYh8IyLrvf9v3RXsmKpKRCJFZJGIrPQey5MB+y67DeUhInGqetg7/Segm6r+MchhVYmIXAx87X18+VkAVX0gyGFViYh0BdzANOBeVa3a2LlB4C2muYkSxTTxvFe07qQb1lAiMhjIAf6lqj2CHU9ViUgLoIWqLhORWGApcFVt/O8iIgLEqGqOiIQBPwB3qepCf3+XXVl4FScKrxhOLE1Sa6jqPFV1emcX4nl7vlZS1fWqujHYcVTR0WKaqloIFBfTrJVUdQFwMNhxnCpV3aOqy7zT2cB6TqxbVyuoR453Nsz7Cci5y5JFCSLyFxFJBa4HHquofS1xG/BFsIOop3wppmmCSETaA32BX4IbSdWJiENEVgD7gS9VNSDHUq+ShYjMF5E1ZXyuBFDVh1W1DTADOGl5kWCr6Fi8bR4GnHiOp8by5VhqKV+KaZogEZEGwMfA3aXuLNQqqupS1T547iAMEJGA3CIM6BvcNY2qDvGx6fvA58DjAQznlFR0LCJyM3A5cJHW8I6pSvx3qW18KaZpgsB7f/9jYIaq/ifY8fiDqmaJyLfAcMDvDyHUqyuLk/GO2lfsCmBDsGI5Vd4CjQ8AV6hqbrDjqccWA51EpIOIhAOjgdlBjqne83YKvwGsV9W/BTueUyEiicVPO4pIFDCEAJ277GkoLxH5GDgDz5M3O4A/ququ4EZVNSKyBYgAiosyLqzFT3aNAF4GEoEsYIWqDgtuVL4TkUuBKRwrpvmXIIdUZSLyAZ6xZxKAfcDjqvpGUIOqAhE5B/geWI3n9x3gIW8tu1pFRHoB7+D5/ysE+FBVJwXkuyxZGGOMqYjdhjLGGFMhSxbGGGMqZMnCGGNMhSxZGGOMqZAlC2OMMRWyZGFMJYhITsWtTrr9RyJymne6gYhME5Gt3oqhC0RkoIiEe6fr1UuzpmazZGFMNRGR7oBDVbd5F03HU5ivk6p2B24BErxFB78Crg1KoMaUwZKFMVUgHs95a1itFpFrvctDROQf3iuFz0Rkjoj8xrvZ9cB/ve06AgOBR1TVDeCtTvu5t+2n3vbG1Ah2mWtM1VwN9AF643mjebGILAAGAe2BnkBTPOWv3/RuMwj4wDvdHc/b6K5y9r8G6B+QyI2pAruyMKZqzgE+8Fb83Ad8h+fkfg7wb1V1q+pe4JsS27QA0n3ZuTeJFHoH5zEm6CxZGFM1ZZUfP9lygDwg0ju9FugtIif7HYwA8qsQmzF+Z8nCmKpZAFzrHXgmERgMLMIzrOU13r6LZngK7xVbD5wOoKpbgSXAk94qqIhIp+IxPESkCZCuqkXVdUDGnIwlC2Oq5hNgFbAS+Bq433vb6WM841iswTNu+C/AIe82n3N88vgd0BzYIiKrgdc5Nt7FBUCtq4Jq6i6rOmv+f3t3iINQEEMBsDUobsTJUAjuQxB4PMdAcQASZBEU+VODADLjVuzal6abPD4sM9dVde/p4BIRm6q6dd/Auc9Li+33G4eI2P5w/zh/xm8o+LxTF9KsImLfE0dU1SMzd/Hq4b4uXe6ipKOg4JuYLAAY2VkAMBIWAIyEBQAjYQHASFgAMBIWAIyeyE+V1D41dowAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x232397f08d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot CV误差曲线\n",
    "test_means = grid_lr.cv_results_[ 'mean_test_score' ] # 每个测试样本得到的分数算均值和标准差\n",
    "test_stds = grid_lr.cv_results_[ 'std_test_score' ]\n",
    "train_means = grid_lr.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = grid_lr.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "\n",
    "# plot results\n",
    "n_Cs = len(Cs)\n",
    "number_penaltys = len(penaltys)\n",
    "test_scores = np.array(test_means).reshape(n_Cs,number_penaltys)\n",
    "train_scores = np.array(train_means).reshape(n_Cs,number_penaltys)\n",
    "test_stds = np.array(test_stds).reshape(n_Cs,number_penaltys)\n",
    "train_stds = np.array(train_stds).reshape(n_Cs,number_penaltys)\n",
    "\n",
    "x_axis = np.log10(Cs)\n",
    "for i, value in enumerate(penaltys):\n",
    "    #pyplot.plot(log(Cs), test_scores[i], label= 'penalty:'   + str(value))\n",
    "    pyplot.errorbar(x_axis, test_scores[:,i], yerr=test_stds[:,i] ,label = penaltys[i] +' Test')\n",
    "    pyplot.errorbar(x_axis, train_scores[:,i], yerr=train_stds[:,i] ,label = penaltys[i] +' Train')\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'log(C)' )                                                                                                      \n",
    "pyplot.ylabel( 'acurracy' )\n",
    "pyplot.savefig('LogisticGridSearchCV_C.png' )\n",
    "\n",
    "pyplot.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "neg-logloss越大越好，logloss有小越好。竖线中心是均值，长度是标准差越长标准差越大"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线性SVM\n",
    "<br/>线性SVM可以用 &nbsp;&nbsp;&nbsp;&nbsp;from sklearn.svm import LinearSVC</br>\n",
    "<br/>或者 &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;from sklearn.svm import SVC  &nbsp;&nbsp;&nbsp;令kernel='linear'</br>\n",
    "<br/>如果只用线性的SVM推荐用LinearSVC会比较快，用核方法的非线性SVM只能用SVC</br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "class sklearn.svm.SVC(C=1.0, kernel=’rbf’, degree=3, gamma=’auto’, coef0=0.0, shrinking=True, probability=False, tol=0.001, cache_size=200, class_weight=None, verbose=False, max_iter=-1, decision_function_shape=’ovr’, random_state=None)\n",
    "\n",
    "degree（多项式核函数的阶数），gamma（δ的倒数）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7654723127035831\n",
      "{'C': 0.01}\n"
     ]
    }
   ],
   "source": [
    "# GridSearchCV 默认的评价指标是正确率\n",
    "\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "\n",
    "param_grid = {'C' : Cs}\n",
    "grid_lsvm = GridSearchCV(SVC(kernel = 'linear'), param_grid , cv=5)\n",
    "\n",
    "grid_lsvm.fit(x_train, y_train)\n",
    "print(grid_lsvm.best_score_)\n",
    "print(grid_lsvm.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "跟逻辑回归差不多"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RBF核的SVM\n",
    "\n",
    "<br/>RBF核的SVM需要调整正则超参数C(正则系数，一般在log域（取log后的值）均匀设置候选参数）和核函数的宽度gamma</br>\n",
    "<br/>C越小，决策边界越平滑</br>\n",
    "<br/>gamma越小，决策边界越平滑</br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7654723127035831\n",
      "{'C': 100, 'gamma': 0.0001}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "Cs = [0.001, 0.01, 0.1, 1, 10, 100, 1000]\n",
    "gammas = [0.0001, 0.001, 0.01, 0.1, 1]\n",
    "\n",
    "param_grid = {'C' : Cs, 'gamma' : gammas}\n",
    "grid_rbf = GridSearchCV(SVC(kernel = 'rbf'), param_grid , cv=5)\n",
    "\n",
    "grid_rbf.fit(x_train, y_train)\n",
    "print(grid_rbf.best_score_)\n",
    "print(grid_rbf.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预测"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "选效果稍微好点的逻辑回归模型做预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[90, 13],\n",
       "       [19, 32]], dtype=int64)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 用训练好的模型分类，用混淆矩阵和f1_score看分类效果\n",
    "from sklearn.metrics import confusion_matrix\n",
    "best_model = grid_lr.best_estimator_\n",
    "y_predict=best_model.predict(x_test)\n",
    "confusion_matrix(y_test, y_predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
